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METHOD AND SYSTEM FOR MEASURING DEGRADATIONS OF A VIDE.O^ > 
IMAGE INTRODUCED BY CODING WITH REDUCTION IN THROUGHPUT 

This invention relates to a method and system for 
measuring degradations of a video image introduced by a 
coding system with reduction in throughput. 

It is particularly but not exclusively applicable to 
5 the domain of low throughput or very low throughput 
digital audiovisual signal distribution networks, and to 
the domain of production of such signals. It is 
particularly applicable to surveillance of the service 
quality of a digital audiovisual signal broadcast 
10 network. 

Digitizing of video signals provides a means for 
copying, storing and transmitting this type of 
information while maintaining a constant image quality. 
However, in practice, the large quantity of information 

15 transferred by video images requires the use of digital 
compression methods to reduce the binary throughput. 

A compression method that is very widely used in 
video is described in standard ISO/CEI 13918 MPEG2 . This 
algorithm is said to be of the "with losses" type since 

20 the restored image after decoding is not identical to the 




original. This algorithm is based on a division of the 
image into blocks and application of a transform, for 
example of the discrete cosine transform type, to the 
pixels in each block to obtain a frequency representation 
5 of the luminance amplitude of pixels in the form of one 
coefficient for each pixel in the block. 

In order to maintain an acceptable quality for the 
final television viewer, the compression algorithms take 
into account perception properties of the human vision 

10 system. However, throughput constraints imposed by 
transmission systems require the application of 
compression ratios that have an influence on the image 
quality perceived by the television viewer. 

It is found that the importance of degradations 

15 caused by coding depends both on the compression ratio 
and the complexity of images. These degradations are 
particularly important when the image is more complex, 
and particularly related to movement of objects, 
brightness and texture. 

20 The degradations that appear in the images following 

application of the MPEG2 coding technique include 
granular errors, deformations of contours, the appearance 
of so-called "exotic" contours and block effects. 

Therefore it would appear necessary to continuously 

25 evaluate the quality of broadcast images. There are 
widely used subjective evaluation methods for this 
purpose, that make use of human evaluation. However, 
these methods are difficult to use and cannot be used on 
a broadcasting network in operation in real time. 

30 There are other so-called "with reference" methods 

based on comparison of the image for which the quality is 
to be evaluated with a reference image. The reference 



image is usually an image that corresponds to the image 
to be analyzed before it is coded and / or transmitted. 
This solution is not very practical, because it requires 
access to one or several reference images. Furthermore if 
5 the video image is transmitted, there is also the problem 
of transporting the reference image to the place at which 
the image to be analyzed is received. 

Other so-called "without reference" solutions are 
used to automatically analyze images without needing to 
10 make a comparison with reference images. The efficiency 
and robustness of each of these solutions lies in the 
method used to measure parameters related to image 
quality. 

Some of these solutions are based on detection of 
15 the block effect made in the spatial domain by gradient 
calculations at block boundaries. To avoid confusion 
between the boundary of an object in the image with a 
block effect, the gradient is compared with intra-block 
gradients. The block effect is detected by means of a 
2 0 decision criterion applied to the behaviour of inter 
block and intra block gradients. 

Thus, the Rohde & Schwarz Company has developed a 
block effect detection method consisting of calculating a 
horizontal gradient vector for each macro block in the 

2 5 image, and calculating an average of each component of 

the vector over the entire image. Variations of 
components of this vector over time are used to bring out 
components with marginal behaviour that represent block 
boundaries degraded by the compression processing 

3 0 transform. Detection of these marginal components 

provides a means for determining a block effect detection 
criterion representative of the image degradation. 



This principle for calculating the gradient is also 
described in patent FR 2 805 429 filed by the Applicant. 
This patent application describes a method based on the 
combination of a binary gradient image and a movement 
5 "pseudo vectors" image calculated from at least two 
successive images. A combination of these two images 
provides a means for estimating a ratio of false contours 
in the image, and is then used to evaluate a quality 
mark . 

10 In patent FR 2 785 116 filed by the applicant, the 

gradients calculated on the entire image to be analyzed 
are passed through psychovisual filters that translate 
the contextual masking effect. A ratio of boundaries of 
detected visible blocks is then calculated searching for 

15 a pseudo periodicity among high value gradients, the 
image quality being evaluated based on this ratio. 

It is found that methods based on the calculation of 
gradients apply filters to only consider a certain type 
of image contents: boundaries or high frequencies. 

2 0 Therefore these methods can only be used to analyze part 
of information contained in the image. The result is that 
they have limited reliability in terms of detection of 
image degradations. Furthermore, methods based on the use 
of gradients to estimate the boundary or for the 

25 detection of contours on the image are relatively 
sensitive to noise, which affects the reliability of the 
estimate of the quality of the intrinsic content of the 
image . 

Furthermore, methods based on the calculation of the 
30 average on the entire image drastically reduce the 
importance of degradations located in a part of the 
image, which makes it difficult to detect such local 



5 



degradations and the arefoire affects the !irelia.bility with 
which the image quality is evaluated. 

Some methods allow an analysis on several successive 
images, to reduce these disadvantages. Therefore these 
5 methods cannot be used to analyze an isolated image 
outside the scope of the video. 

The purpose of the present invention is to eliminate 
these disadvantages. This purpose is achieved by 
providing a method for measuring degradations of a 

10 digitized image, introduced when coding the image, the 
method consisting of dividing the image into coding 
blocks using a coding grid and applying a coding 
processing on pixel data in each block, making use of a 
block transform calculation and an inverse block 

15 transform calculation. According to the invention, this 
process includes steps of : 

- determining the coding grid of the coded image, in 
order to find the image division into coding blocks, 
used when coding the image, 

20 - shifting the coding grid with respect to the coded 
image, so as to define an image division into analysis 
blocks each covering a boundary between two adjacent 
coding blocks, 

- applying the block transform calculation to pixel data 
2 5 in the coded image using the shifted coding grid to 

obtain transformed coefficients for each analysis block 
defined by the shifted coding grid, 

- extracting coefficients that could be affected by a 
block effect resulting from coding of the image, from 

30 the transformed coefficients. 



- applying the inverse block transform calculation to the 
extracted transformed coefficients to determine the 
pixel data for each analysis block, 

- for each analysis block, estimating an indicator of the 
5 degradation due to block effects, using pixel data in 

the coded image and pixel data in each analysis block, 
obtained by the inverse transform calculation, and 

- determining an image degradation measurement by summing 
the degradation indicators of each analysis block . 

10 According to a special feature of the invention, the 

estimation of a degradation indicator for each analysis 
block comprises steps of : 

- calculating an average of inter pixel differences at 
the inter block boundary of the coding grid, covered by 

15 the analysis block, using pixel data obtained for the 

analysis block, 

- calculating an average and a standard deviation 
applicable to pixels in the two adjacent blocks on the 
coding grid, partially covered by the analysis block, 

20 — calculating a weighting factor as a function of the 
average and the standard deviation obtained for the 
analysis block, and 

- calculating a spatial activity of the analysis block 
using spatial activities determined for each of the two 

25 adjacent blocks in the coding grid partially covered by 

the analysis block, 

- the analysis block degradation indicator being 
determined as a function of the calculated average of 
inter pixel differences, the weighting factor and the 

30 spatial activity of the block. 
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Advantageously, the analysis block degradation 
indicator is obtained using the following formula: 



^i, j 



l + v|/|ACTij| 

in which Ali,j is the average of inter pixel differences 
5 at the inter block boundary of the coding grid covered by 

the analysis block, Wi,j is the weighting factor, V|/ is a 

predefined constant, and ACTi,j is the spatial activity of 

the analysis block. 

According to another special feature of the 
10 invention, the transform calculation is applied to coding 

blocks of the coded image, the spatial activities 

determined for each of the two coding blocks being 

obtained from the transformed coefficients for each of 

the two coding blocks . 
15 Advantageously, the spatial activities determined 

for each of the two coding blocks are obtained from the 

following formulas : 



ACTifj= ^—-r 

1 + DCp. 



i;[AClfj(u,v)Nill(u,v)f 



u,v=0 
u + V9tO 



ACTPj = ' 



1 + DCPj 



1 



X[ACPj(u.v)Nill(u,v)f 



u,v=0 
u+v=?tO 



20 in which DCy and ACy (u,v) and DCjj and ACy (u,v) are 

the transformed coefficients for each of the two adjacent 
coding blocks partially covered by the analysis block, 
and Nill(u,v) is a masking function modelling masking by 
neighbourhood . 

25 Preferably, the average and standard deviation 

calculated for each analysis block are determined from 



transformed coefficients for each of the two adjacent 
coding blocks partially covered by the analysis block. 

According to another special feature of the 
invention, the weighting factor is obtained by the 
5 following formula: 



and ai,j being the average and standard deviation 
respectively calculated for each analysis block and ^ 

10 being a parameter corresponding to the maximum 
sensitivity of the human eye . 

According to another special feature of the 
invention, analysis blocks that could contain a block 
effect are selected before estimating a degradation 

15 indicator for each analysis block. 

Advantageously, the prior selection comprises a step 
consisting of separating analysis blocks for which the 
extracted transformed coefficients are greater than a 
predetermined threshold. 

2 0 Preferably, the prior selection comprises a step 

consisting of selecting analysis blocks with pixels with 
an energy representing a significant proportion of the 
energy of the block, at the inter block boundary of the 
coding grid covered by the analysis block. 



Wi,j (Mi,j,C7i,j,^) = 




in which X 
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According to another special feature of the 
invention, the coding grid is shifted horizontally with 
respect to the coded image. 

Alternately, the coding grid is shifted vertically 
5 with respect to the coded image. 

Preferably, the block transform calculation is a 
discrete cosine transform calculation. 

The invention also relates to a system for measuring 
degradations of a digitized image introduced when coding 
10 the image, the system comprising calculation means for 
implementing the method defined above. 

One preferred embodiment of the invention will be 
described below as a non- limitative example with 
reference to the appended figures, wherein: 
15 Figure 1 diagrammatical ly represents the measurement 

system according to the invention, integrated into an 
image processing system; 

Figure 2 shows the measurement system represented in 
Figure 1 in more detail; 
2 0 Figure 3 shows two image pixel blocks to illustrate 

the method according to the invention, 

Figure 4 shows part of the system shown in Figure 2 
in more detail; 

Figure shows a curve representing a weighting 

2 5 function used by the method according to the invention; 

Figure '6 shows other curves illustrating another 
weighting function used by the method according to the 
invention; 

Figure 7 shows curves illustrating the variation in 

3 0 degradation measurements obtained using the method 

according to the invention applied to a sequence of 
images . 
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Figure 1 represents a degradation measurement system 
1 according to the invention that is designed for use as 
a probe that could be applied to any point in a digital 
image processing system such as broadcasting system or a 
5 video signal production system. 

The measurement system 1 is designed particularly to 
measure the quality of images output from a coding system 
10 with reduced throughput respecting the MPEG 2 
standard. For this purpose, it is based on a frequency 
10 and time filtering combination. 

This system is applicable whenever there is a need 
to identify coding defects in a digital video signal, 
particularly to determine the appropriate throughput for 
a given image sequence as a function of the expected 
15 quality. 

The coding principle according to the MPEG 2 
standard consists of dividing the digitized image into 
blocks of N X N pixels (for example where N is equal to 
8) using a coding grid, and applying a discrete cosine 

20 transform (DCT) to each block to change from the spatial 
domain to the frequency domain, and then to cancel some 
components of the transform corresponding to high 
frequencies before applying the inverse DCT 
transformation to change back to the spatial domain, in 

25 other words so as to recover the pixels from the 
corresponding block. 

The discrete cosine transform processing consists of 
making a calculation of transform coefficients AC and DC 
obtained from the following formulas, to each block Bi,j 

30 of N X N and pixels in the image: 

2 N-lN-l 
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and 

ACi,j(u,v) = Fi,j(u,v), where u + v o (2) 



in which 




(3) 



x=Oy=0 V V J J 

5 fi,j(x,y) represents the luminance of the pixel at point 
(x,y) in block Bi,j, x and y being the horizontal position 
index and the vertical position index respectively of the 
pixel in the block Bi,j, and u and v are between 1 and 
N-1 and represent the indexes of the horizontal and 

10 vertical spatial frequencies respectively, and c(0) = ^= 

V2 

and c (u) = 1 if u 9^ 0. 

The inverse processing of the discrete cosine 
transform consists of applying the following formula to 
the coefficients DCi,j and ACi,j(u,v) of each block Bi ji 

'm ••' 

In Figure 2, the measurement system 1 comprises a 
synchronisation module 11 that synchronises each coded 
image to be processed with its coding grid, in other 
words the module 11 divides the image into blocks of 

20 N X N pixels. One example of a synchronisation processing 
that could be done by this module is described in patent 
application FR 2 769 452 filed by the Applicant. This 
synchronisation processing identifies the division into 
blocks used by the previous coding processing done by the 

25 system 10, within the image, and therefore determines the 
coding block Bi,j to which each pixel in the image 
belongs . 



15 li, j (x, y) = — 2 Z c(u)c(v)Fi J (u, v) cos l^^^l^-^^jj ^osl ny 



The coded image and its division into blocks are 
applied to two processing branches 2, 3, the first branch 
2 comprising a discrete cosine transform module 16, and 
the second branch 3 comprising firstly a shift module 12 
5 producing a horizontal shift of the coding grid by half a 
block with respect to the image. Figure 3 illustrates 
this horizontal shift by half a block, by showing a block 
Bi,j of 8 X 8 pixels horizontally shifted by half a block 
with respect to the coding grid illustrated by the 

10 adjacent blocks 21 and 22. 

In the second processing branch 3 , the coded image 
and the shifted decoding grid at the output from the 
shift module 12 are applied to the input of a discrete 
cosine transform module 13. A horizontal frequencies 

15 extraction module 14 then processes transform 
coefficients output from module 13- In fact, this module 
consists of assuring that the frequency domain only 
retains frequencies (coefficients ACi,j(u,v) and DCi,j} 
that could make a vertical block effect visible. If 

20 N = 8, it could thus be estimated that it is only 
necessary to retain coefficients DCi,j, ACi,j(u,v) such 
that : 

0<u<8ifv=0, 
2<u<8ifv=l, and 
25 6<u<8ifv=2 

The retained coefficients ACi,j(u,v) and DCi^j are 
then processed by an inverse DCT transformation module 15 
to retrieve spatial information Ii,j(x,y) = I(8i+x, 8j+y) 
30 related to the retained frequencies obtained using 
formula (4) . In this respect, it is possible to consider 
only the fourth and fifth columns 24, 25 concerned by the 
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block effect due to coding, and which correspond to the 
boundary 23 between the blocks 21 and 22 of the coding 
grid in Figure 3 . 

Therefore, the output from branch 3 provides pixel 
5 data I(8i+x, 8j+3) and I(8i-i-x, 8j+4) for the complete 
image, where i and j are the indexes of shifted blocks of 
the image using the shifted coding grid obtained at the 
output from the shift module 12. The output from branch 2 
provides coefficients DCi^j and ACi,j(u,v) for each image 
10 block related to the coding grid at the output from the 
synchronisation module 11. For each shifted block Bi,j, 
the coefficients DC?- and ACy (u,v) in the left block 21 

and the coefficients DC^ and AC^j (u,v) of the right 
block 22 are available. 
15 The data output from the two processing branches 2 

and 3, in other words at the output from modules 16 and 
15, are applied to the input of a block effect evaluation 
module 17. 

The data at the input to module 17 shown in Figure 4 
20 are applied to the following modules: 

- a module 31 for calculating the average of inter pixel 
differences at the vertical boundary 23 between each 
pair of adjacent blocks 21, 22 in the coded image; 

- a module 32 for calculating the average and the 
25 standard deviation of the luminance of each block; and 

- a local contrast calculation module 34. 

The module 31 applies the following formula to pixel 
data output from the second branch 3 , to evaluate the 
average of inter pixel differences on each shifted block 
30 Bi,j: 
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^i^D = I(8i4-k,8j + 4)-(8i + k,8j4-3)|| (5) 

The average thus calculated actually describes the 
behaviour of the gradient operator at the vertical 
boundary 23 of two adjacent blocks 21, 22 output from the 
5 image coding grid. 

In module 32, an average /ii^j and a standard 
deviation ai,j of the luminance are estimated for each 
block Bi,j on the two corresponding adjacent blocks 21, 22 
in the coding grid. For example, the AC and DC 
10 coefficients of the coding blocks in the DCT domain 
output by the first branch 2 could be used according to 
the following formulas: 

Mi.j = i(DC°j+DC°) (6) 

and 

r 



j- i:[ACa(u,v)f + i i[ACPj(u,v)f 



(7) 



U + V7tO y U + V9feO 

Obviously, the average and the standard 

deviation ai^j could also be obtained directly from the 
pixels of blocks 21, 22 corresponding to each block Bi^j 
using classical formulas for calculating the average and 

2 0 the standard deviation. 

The averages and standard deviations calculated by 
the module 32 are then applied to a module 33 for 
calculation of a weighting factor Wi,j()Lti^j, ai,j, Q for 
each block Bi,j. The module 33 applies the weighting 

25 function defined by the following formula to calculate 
this weighting factor: 
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Wi,j(Ati,j/ CTi.j, Q = 



A.ln 



1 + - 



1 + CT 



■•V 



In 



else. 



(8) 



In 



where 



'^j J 



In 



1 + 



(9) 



and ^ is a parameter corresponding to the maximum 
sensitivity of the eye. 
5 This weighting function is described in the 

publication "A Generalized Block-Edge Impairment Metric 
for Video Coding" by H.R Wu and M. Yuen, IEEE Signal 
Processing Letters, Vol. 4, No. 11, November 1997. 
However, the weighting function described in this 

10 publication has a discontinuity at point /xi^j = when the 
standard deviation ai,j is not zero. Therefore, this 
function is adapted in the method according to the 
invention to be continuous at this point. 

Figure 5 illustrates this weighting function in the 

15 form of a curve 41 as a function of the grey level or the 
average luminance of pixels in the block, in the case 

in which ^ is chosen to be equal to 81, corresponding to 
the maximum sensitivity of the human eye, and where a is 
equal to 20. 

20 The local contrast calculation performed by module 

34 consists in estimating the spatial activity ACTi,j of 
each block Bi^j in the transform domain, starting from 
coefficients DCi,j and ACi,j, the coefficients ACi^j being 
weighted by a masking function to model masking by the 



16 



10 



15 



20 



neighbourhood. The spatial activity ACTi,j of each block 
Bi,j is thus estimated by applying the following formula: 



ACTi,3 = i(ACT°+ACTi;]) 



where 



ACTfl = ^ 



l + DCS 



X[ACjfj(u,v)Nill(u,v)f 



u,v=0 



and 



ACT^^- = ' 



1 + DCPj 



X[AC,^j(u,v)Nill(u,v)f 



(10) 



(11) 



(12) 



u,v=0 
u + V5itO 



in which Nill(u,v) is the masking function that can for 
example be the function described in the document "A 
Visual Model Weighted Cosine Transform for Image 
Compression and Quality Assessment" by N. B. Nill, IEEE 
Transactions on Communications, Vol. COM-33, No. 6, June 
1985 . 

In this document, the masking function is given by 
the following formula: 

Nill(u,v) = A(a))H(o)) (13) 



where g>= represents the radial frequency expressed 

in cycles per degree (u and v e[0,N-l]), 



A{a>) = 



1 1 



In 



27171 



a 



47171^00 



-l2 



+ 1 



H(co) = (0.31 + 0.69(0)e 



-0.29© 



(14) 



(15) 



and a is commonly equal to 11.636 deg'^ for processing of 
luminance on 8 bits. 

Figure 6 illustrates this weighting function by 
representing the curve 43 showing the variation of U{(o) , 
the curve 44 showing the variation of A(co) and the cuirve 
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45 showing the variation of the function Nill (co) , as a 
function of the spatial frequency co . 

In formulas (11) and (12) , normalisation by the DC 
coefficient provides a means for considering the match to 
5 the local contrast in accordance with the Weber's law, 
and one is added to the denominator so that the function 
is continuous at zero, 

Finally, the results of calculations carried out by 
modules 31, 33 and 34 are applied to a module 35 for 
10 evaluation of the image degradation u due to the edge 
effect resulting from coding. This module carries out the 
following calculation to estimate an image degradation, 
measurement : 

o = ^Dy (16) 
15 where Ui,j = p (17) 

l + V|/ACTy| 

represents the impact of block effects on block Bi^j and 
\|/ is an experimentally determined constant that adjusts 
the importance assigned to the spatial activity compared 
with the weighted average of inter pixel differences. 

20 The measurement of the degradation u estimated by 

this module corresponds to an indicator of the visibility 
by the human eye of degradations due to the block effect. 

Figure 7 shows variations of this measurement in the 
shape of curves 46, 47, 48, as a function of the image 

25 number in a sequence of video images, for throughputs of 
9, 6 and 4 Mbits/s respectively. 

These curves show that image degradations are 
amplified as the throughput is reduced. Therefore, the 
coarse coding by quantification of DCT coefficients 
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results in the block effect- The part of the image 
sequence in which the degradation measurement is high 
corresponds to images for which the spatial and time 
cont ent s change suddenly . 
5 Processing done by module 17 can be optimized by 

submitting input block data to a selection using a module 
for selection of blocks Bij that could contain a block 
effect. To achieve this, this module compares the energy 
of central columns 24 and 25 of each block Bij with 

10 energy of the block. The block is selected to estimate 
the degradation if the energy of columns 24 and 2 5 
represents a significant proportion, for example more 
than 25%, of the total energy of the block. Before this 
selection is made, blocks for which the DCT coefficients 

15 selected by the extraction module 14 are below a 
predefined threshold, for example close to zero, can be 
eliminated - 

Note that the processing that has just been 
described is designed to detect a vertical block effect. 

20 Obviously, this processing could be modified in an easily 
understood manner to detect a horizontal block effect. 
All that is necessary in module 12 is to make a vertical 
shift of a half block, and then in the frequency 
extraction module 14 to select frequencies that could be 

25 affected by a horizontal block effect, and finally in 
module 15 to consider all of the reconstituted pixels and 
only retain pixels in the fourth and fifth lines of the 
shifted block corresponding to the horizontal boundary 
between the two adjacent non-shifted blocks corresponding 

30 to the shifted block. 

It would also be possible to perform vertical and 
horizontal block effect detection processings in 



19 



parallel, and to use the results output by these two 
processings at the same time, for example in module 35, 

After comparing objective measurements obtained by 
the method and the system according to the invention, 
5 with marks assigned during subjective tests by panels of 
representative persons, it is found that there is a 
strong correlation between objective measurements and 
subjective measurements, confirming that the method and 
system according to the invention are efficient. 

10 



